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We present a search for charged Higgs bosons in decays of top quarks, in the mass range 80 < 
mif± < 155 GeV, assuming the subsequent decay H^ — > T^r^r (and its charge conjugate). Using 
0.9 fb~^ of lepton+jets data collected with the DO detector at the Fermilab Tevatron pp collider, 
operating at a center of mass energy -^s = 1.96 TeV, we find no evidence for a H^ signal. Hence 
we exclude branching ratios B{t H+b) > 0.24 for m^± = 80 GeV and B{t H+b) > 0.19 for 
m„± = 155 GeV at the 95% C.L. 

PACS numbers: 14.65.Ha, 12.60.Fr, 14.80. Cp 



The electroweak symmetry breaking sector of the stan- 
dard model (SM) contains a single SU(2) complex scalar 
doublet field that provides gauge-invariant generation of 
particle masses, with the only observable particle being 
the electrically neutral Higgs boson H° li. Here, we 



search for evidence of a richer structure. The simplest 
extension to the SM Higgs sector involves the addition 
of a second SU(2) complex scalar doublet, which intro- 
duces five spin-0 particles, three that are neutral and two 
that are charged (H^) 0. The fermion couplings to the 
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Higgs doublets are not specified a priori, and the only re- 
quirement is that flavor changing neutral currents are not 
allowed at lowest level in perturbation theory. One pos- 
sibility, the Type-II model, couples the up-type fermions 
to one Higgs doublet and the down-type fermions to the 
other, as required in the minimal supersymmetric exten- 
sion to the SM (MSSM) 0]. In addition, the MSSM 
constrains the five Higgs masses through two free param- 
eters: tan/3, the ratio of the vacuum expectation values 
of the two doublets, and the mass of any one of the phys- 
ical Higgs bosons. We choose to^± for the latter. 

Since the Yukawa coupling to the boson increases 
with fermion mass for all values of tan /3, top and bottom 
quarks in this model are expected to have large Yukawa 
couplings. Therefore, if mfj± < nit — mh, the decay 
t — > H^b (and its charge conjugate) is expected to have 
a large branching fraction for all tan/3. Further, for large 
values of tan/3 (tan/3 > 10), the charged Higgs decays 
predominantly to a t lepton and its associated neutrino 
with B[H^ t^i^t) ~ 1- Hence if the boson exists 
and B{t H^b) is substantial, a search optimized for 
the study of SM decays of tt to lepton-|-jets final states 
should show a deficit of events relative to the SM predic- 
tion because of differences in decay branching fractions 
and in kinematic distributions. Any such deficit could 
therefore be indicative of the presence of charged Higgs 
bosons in decays of the top quark. 

Direct searches for bosons have been performed 
at the LEP e+e" collider at CERN ^ and at the Teva- 
tron pp collider at Fermilab Q]. With no evidence of 
a signal, the LEP experiments set a combined limit of 
mH± > 78.6 GeV independent of B{H^ — > r+i/r), while 
the Tevatron experiments have set limits in the context 
of a Type-H two Higgs doublet model that exclude re- 
gions of the [tan/3, m[j±] parameter space Searches 
for indirect evidence of bosons through radiative de- 
cays of B mesons at B factories provide a combined limit 
of mH± > 295 GeV [1, 0, H- Although B factories ex- 
clude a larger part of parameter space than our current 
study, it is important to search for objects such as the 
bosons through all possible channels and not defer 
entirely to theory. 

In this article, we describe the search for charged Higgs 
bosons from top quark decays in tt events with one lepton 
(electron e or muon /i) and at least three jets. A repre- 
sentative Feynman diagram for such events is shown in 
Fig. [TJ where one of the top quarks decays to a boson 
and a b quark, as in the SM, and the other decays to a 
boson and a b quark. For our signal, we consider 
events in which the W boson decays leptonically (e, /i, 
or T, with the r decaying to an e or /i and two neutri- 
nos), while the charged Higgs boson decays to a r and 
a neutrino and the r decays to a neutrino and hadrons. 
The final state therefore consists of an isolated lepton (e 
or fi) with large transverse momentum (jdt), significant 
missing transverse energy (^t) from the escaping neu- 



trinos, and at least three jets: two from the b quarks 
and one from the decay of the r. No attempt is made 
to identify r leptons in such decays. Some of the signal 
can also come from events where the r from the bo- 
son decays leptonically, while the W boson decays into 
a quark-antiquark pair, thereby giving two jets. In that 
case, there will be four jets in the final state. Finally, if 
both top quarks decay into charged Higgs bosons, which 
then decay into r leptons, and one r decays leptonically 
while the other decays into a jet, this can also contribute 
to the signal. The largest backgrounds to these processes 
are from SM decays of tt pairs and H^-|-jets production, 
along with smaller contributions from the production of 
single top quarks, dibosons {WW, WZ, and ZZ), and 
Z-|-jets. An additional source of background is from mul- 
tijet events, in which a jet mimics an electron, or a muon 
from b (or c) quark decay appears to be isolated. 





FIG. 1: Representative Feynman diagram for charged Higgs 
boson production in top quark decays at the Tevatron {£ — e 
or n). 

We analyze 0.90 ± 0.05 fb^^ of data recorded with 
the DO detector 0, The trigger required a recon- 

structed jet and an electromagnetic energy cluster in 
the electron channel or a jet and a muon candidate in 
the muon channel. We base this analysis on a pre- 
vious one that extracted the tt production cross sec- 
tion within the framework of the SM, i.e., assuming 
B{t ^ W^b) = 1 [llj. The principal difference is that 
here we consider an additional decay mode (t — > H^b) 
and attempt to measure B = B{t ^ H'^b) under the 
constraint B{t ^ W+b) + B{t ^ H+b) = I. For any 
measurement of B, mu± is treated as a fixed parameter. 
Measurements are made for several values of m//± . 

We apply the same event selection criteria as in 
Ref. [11| to separate ti production from background. 
These are summarized in Table HI We impose an ad- 
ditional requirement of X]PT(jet) > 120 GeV for events 
with only three jets and separate the events into two jet- 
multiplicity bins (3 jets and > 3 jets) to improve signal 
discrimination. 

To model the background distributions, W+jets and 
Z-fjets events are generated using ALPGEN [13l|, while 
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TABLE L Summary of event selections. 





e + jets channel 


fj, + jets channel 


Lepton 


w 


PT > 20 GeV 


PT > 20 GeV 






\rj\ < 1.1 fl2] 


|r,| < 2.0 






> 20 GeV 


> 25 GeV 




?t) ^2] 


> 0.7tv - 0.045^T 


> 2.l7r - 0.033^T 


(^T in 


GeV) 






Jets 




> 2,pt> 20 GeV, \rj\ < 2.5 






PT(jetl) > 40 GeV 



SINGLETOP is used for single top quark events. The 
events are passed through pythia \15^ for parton show- 
ering and hadronization. Diboson and SM tt events 
are generated using pythia. The non-SM decay modes 
of tt events where one or both top quarks decay to a 
charged Higgs boson, are also generated using pythia. 
The Monte Carlo (MC) events for the signal are pro- 
duced at the following values of m/^±: 80, 100, 120, 140, 
150, and 155 GeV. All MC generated events are processed 
through the DO detector simulation based on GEANT 
followed by application of the same reconstruction algo- 
rithms as used on DO data. Subsequent corrections are 
also applied to MC events to account for trigger efhcien- 
cies and differences between MC events and data in ob- 
ject reconstruction efficiencies and resolutions. 

To determine the number of background multijet 
events, we use a data sample with looser electron iden- 
tification or weaker muon isolation criteria, as described 
ll|. The normalization of the VF-|-jets contri- 
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bution is determined differently in the current analysis, 
as discussed below. For the prediction of yields for the 
single top quark, diboson, and Z 4- jets events, we use 
next-to- leading order cross sections [17[. The number of 
tt events is obtained by summing the different top quark 
decay modes according to their accepted branching frac- 
tions and respective selection efficiencies (e) as follows: 

Nti = [(1 - Bf ■ eww + 2(1 - B)B ■ ewH 

+B^ ■ eHH] ■ a{tt) ■ j Cdt, (1) 

where WW represents SM decays of the top quark, WH 
and HH represent non-SM decays of one or both top 
quarks, respectively, and / Cdt is the integrated lumi- 
nosity. We use cr{tt) — 7.48l^J^ pb for a top quark mass 
of mt = 172.4 ± 1.2 GeV |li|, and consider B as the 
parameter of interest for any fixed value of m//± . The 
selection efficiencies for the WW decay modes in the dif- 
ferent channels are « 2%, which includes all corrections 
and trigger effects. The corresponding efficiencies for the 
WH {HH) modes vary between 1.5%-0.5% (1.2%-0.3%) 
for different values of mfj± . 

To differentiate between tt and background, we define 
a multivariate discriminant 



where p is the probability density for a set of observed 
variables x, given the signal (S) or background {B) class 
of events. The signal comprises tt events which include 
decays of the top quark and hence depends on the 
values of B and mfj± ■ For the construction of the dis- 
criminant, we define the signal for only one value of B 
that corresponds to the SM scenario [B — 0). The back- 
ground is defined by all non-tt events. The variables 
for the different final states are listed in Table [ill The 
normalization for the VF-l-jets template is obtained from 
the low-P region (2? < 0.45), which is background dom- 
inated, by setting the sum of all backgrounds and signal 
in this region to the corresponding observed number of 
events. We recompute the normalization for each value 
of B and mH± ■ The number of predicted (and observed) 
events for the full range of T> appears in Table IIIII for 
B — Q. The corresponding distributions are shown for 
toh± = 120 GeV in Fig. [2] for e + > 3 jets, for S = 
and B = 0.5, in (a) and (b), respectively. We see that the 
data agree well with the SM predictions. Similar agree- 
ment is seen in all other channels. Hence, we proceed 
to set upper limits on the non-SM branching fraction 
B[t H+b). 

TABLE II: Variables used to define the discriminant T>. AR = 
(A(^)2 -I- (Ar;)2 and i indexes the list of Nj jets ordered in 
decreasing pt- 



Variable 



Channel 



E£iPT(i)/ESiP4i) 

AR{£,jetl) 
A7?(jetl,jet2) 
A<j,{£, ^t) 
A0(jetl, ^t) 
Sphericity S [19] 
Aplanarity A [19] 



all 

e -f 3 jets, e -I- > 3 jets 
't e + 3 jets, e + > 3 jets 
all 

e + > 3 jets, /i + > 3 jets 

4- 3 jets, /I > 3 jets 
e -I- 3 jets, p -I- 3 jets 
all but /i -I- 3 jets 
all but ^ -|- 3 jets 



2?(x) 



p(x|5) 



p(x|5)+p(x|Z?)^ 



(2) 



We use a modified frequentist approach to set lim- 
its at the 95% C.L. in the high-2? region since it is tt dom- 
inated. Sources of uncertainty on the predicted yields are 
included with correlations across samples and channels. 
Their estimated values are provided in Table IIVI Note 
that all these uncertainties are applied to the VF-|-jets 
normalization assuming full anticorrelation because of 
the manner in which the W^-|-jets normalization is derived 
as explained above. The dominant sources of uncertain- 
ties are from the integrated luminosity, the jet energy cal- 
ibration, and the tt cross section. The uncertainties from 
the normalization of multijet, single top quark, diboson, 
and Z-|-jets events have a less pronounced effect on the B 
limits because of the smaller contribution of these sam- 
ples in the high discriminant region. We consider the 
distribution in V above 0.55 in the > 3 jets channels and 
0.6 in the 3 jets channels, a choice determined by maxi- 
mizing the sensitivity of the analysis in MC simulation. 
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TABLE III: Event yields after all selections for channels separated by lepton flavor and jet multiplicity. We assume B{t — » 
H^b) = so that ti includes only SM decays of the top quarks. The "other MC" comprises single top quark, diboson, and 
Z+jets events. (The uncertainty on the total SM prediction includes correlations across samples.) 



Source 


e ± 3 jets 


/I ± 3 jets 


e + > 3 jets ^ ± > 3 jets 


Signal {tt) 


148.8 ±20.0 


108.2 ±14.7 


130.4 ±19.4 


105.6 ±15.4 


W+jets 


535.4 ±47.9 


572.4 ±34.7 


79.2 ±17.3 


152.0 ±16.5 


Other MC 


102.5 ±14.6 


106.7 ±15.3 


33.1 ±4.8 


35.0 ±5.3 


Multijets 


194.2 ±30.5 


33.5 ±13.9 


60.2 ±10.1 


10.4 ±5.7 


Total SM prediction 980.9 ±25.8 820.8 ±27.6 302.9 ±13.1 


303.0 ±15.6 


Observed 


948 


812 


320 


306 




D0 0.9 fb-^ 
tf (B = 0) 
Other MC 
W+jets 
Multijet 



(a) 



e+>3jets 




0.6 0.8 1 
Discriminant 




D0 0.9 fb"^ 
tt (B = 0.5) 
Other iUIC 
W+jets 
lUlultijet 



(b) 



e+>3jets 



0.6 0.8 1 
Discriminant 



FIG. 2: Distributions in the 
mu± = 120 GeV in e ± > 3 jets, 
and (b) B = 0.5. 



discriminant V for 
for (a) B = (SM), 



The sensitivity is defined as the median of limits obtained 
from an ensemble of background plus SM tt {B = 0) 
pseudo experiments in each channel. We call these the 
expected limits and show them by the dashed curve in 
Fig. [3] along with their ±1 standard-deviation (SD) in- 
tervals by the cross-hatched region. The observed limits, 
using DO data, are shown by the solid curve in Fig. [3l 

The upper limit on B{t H'^b) can be used to ex- 
clude regions of the [tan f3, toh±] parameter space in the 
context of the MSSM. Since the MSSM has several free 
parameters, we select them according to the m'^'^^ sce- 



nario described in Ref . [21| . This provides the maximum 
range in the mass of the lightest neutral Higgs boson as 
a function of tan (3. The exclusion bounds are calculated 



TABLE IV: Uncertainties (equivalent to ±1 SD) from differ- 
ent components affecting the predicted yields. "Other MC" 
comprises single top quark, diboson, and Z±jets events. 



Component 


Uncertainty [%] 


Integrated luminosity 


6.1 


Primary vertex modeling 


2.2 


Trigger efficiency 


0.5-2.8 


Lepton identiflcation 


2.2-2.6 


Jet energy calibration 


5.0 


Jet identiflcation 


2.0-2.4 


Jet energy resolution 


0.1-1.8 


Multijets normalization 


15.7-54.8 


Other MC normalization 


11.0-12.0 


a{tt) 


7.4-9.6 


mt 


2.1 


MC statistics 


0.9-25.0 



T 



0.5 
0.45 
0.4 



0.35 

c 

0.3 



o 
*.» 

E 



0) 
Q. 
Q. 



0.25 
0.2 

0.15 
0.1 

0.05 




DO 0.9 fb"^ 



^ Expected ± 1 SB 
Observed 




80 



100 



120 



140 



160 
[GeV] 



FIG. 3: The 95% C.L. limits on B{t ~* H+b) for different 
values of m^fi. 

using FeynHiggs [2§|, which includes the two- loop QCD 
and MSSM corrections. Figure H] shows the expected and 
observed excluded regions, and the theoretically inacces- 
sible region defined as the boundary where certain Higgs 
parameters acquire unphysical values. 

In summary, we have analyzed 0.90 ± 0.05 fb^^ of lep- 
ton+jets data at DO, and found no evidence for top quark 
decays to charged Higgs bosons. Hence we set upper lim- 
its at the 95% C.L. on B{t H+b) ranging from 0.24 
for mH± = 80 GeV to 0.19 for mjj± = 155 GeV. 
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